Skip to content

Parallelize release builds - #1930

Merged
adamchalmers merged 6 commits into
mainfrom
codex/shard-release-builds
Oct 27, 2026
Merged

adamchalmers merged 6 commits into
mainfrom
codex/shard-release-builds

Conversation

@adamchalmers

@adamchalmers adamchalmers commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Background: We publish the CLI for five OS/arch combinations: Mac (x86, ARM), Windows (just x86) and Linux (x86, ARM).

Problem: The MacOS runner builds two binaries in serial (x86, then ARM). The Ubuntu runner builds three binaries in serial (Windows x86, Linux x86, Linux ARM). This is fine, but slow. It'd be faster to parallelize across 5 runners, one per job. That's how the check-release CI job works, we should make the real release work the same way.

Solution: Prepare Cargo sources, the Linux cross executable, version, and prerelease status once, then run one shard per target. So, there's a shared initial "prepare" step, then shard across the five targets, then come back together to verify/document/publish them.

prepare
  ├─ macOS x86-64
  ├─ macOS ARM64
  ├─ Windows GNU
  ├─ Linux x86-64 musl
  └─ Linux ARM64 musl
       ↓ all five finish
  verify assets + combine instructions → publish → Homebrew

Share preparation with PR CI through a composite action. Reuse PR build caches, scoped by host OS/architecture, target, Rust compiler, and lockfile. Collect artifacts before one upload; preserve binary names, checksum files, and download URLs. Homebrew uses the prepared version without installing Cargo tools or compiler packages.

Testing

  • Before: The successful release spends 23m17s building three Linux/Windows targets serially and 16m38s on two macOS targets. Homebrew's dependency installation takes 46m57s.
  • After: A release dry run passes all five target builds in parallel (3m7s–4m7s), following 27s of shared preparation. Sources, the cross executable, and all five compiled build caches restore successfully. These are warm-cache timings, not a controlled speedup benchmark.
  • Artifact collection downloads all five artifacts and verifies both checksums for every binary before assembling the combined instructions and download directory.
  • Homebrew packaging passes in 20s without dependency installation, using the existing published binaries. Publishing, cloud authentication, and Homebrew repository updates were disabled in this dry run.
  • Local checks cover stable/rc/pre version outputs, the 15 release assets plus combined README, and missing/corrupted artifacts failing before publication. Workflow lint and Bash syntax checks pass.
  • Temporary validation wiring has been removed. Existing required PR check names are unchanged.

Merge Danger

Door: Two-way

Blast Radius: Releases

Changes release scheduling, artifact collection, and publishing dependencies. Publishing waits for every target and checksum verification. No new tag or release was published during validation.

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.20%. Comparing base (bf645ee) to head (9fffb07).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1930   +/-   ##
=======================================
  Coverage   68.20%   68.20%           
=======================================
  Files          43       43           
  Lines        8743     8743           
=======================================
  Hits         5963     5963           
  Misses       2780     2780           
Flag Coverage Δ
unittests 68.20% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adamchalmers
adamchalmers marked this pull request as ready for review October 27, 2026 13:57
@adamchalmers
adamchalmers force-pushed the codex/shard-release-builds branch from 3bb53a5 to 9fffb07 Compare October 27, 2026 17:46
@adamchalmers adamchalmers changed the title Build release targets in parallel with shared preparation Parallelize release builds Oct 27, 2026
@adamchalmers
adamchalmers merged commit 6093a6e into main Oct 27, 2026
29 checks passed
@adamchalmers
adamchalmers deleted the codex/shard-release-builds branch October 27, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants